Diagnostic accuracy of virtual cognitive assessment and testing: Systematic review and meta-analysis.

BACKGROUND/OBJECTIVES: Virtual (i.e., telephone or videoconference) care was broadly implemented because of the COVID-19 pandemic. Our objectives were to compare the diagnostic accuracy of virtual to in-person cognitive assessments and tests and barriers to virtual cognitive assessment implementation. DESIGN: Systematic review and meta-analysis. SETTING: MEDLINE, EMBASE, CDSR, CENTRAL, PsycINFO, and gray literature (inception to April 1, 2020). PARTICIPANTS AND INTERVENTIONS: Studies describing the accuracy or reliability of virtual compared with in-person cognitive assessments (i.e., reference standard) for diagnosing dementia or mild cognitive impairment (MCI), identifying virtual cognitive test cutoffs suggestive of dementia or MCI, or describing correlations between virtual and in-person cognitive test scores in adults. MEASUREMENTS: Reviewer pairs independently conducted study screening, data abstraction, and risk of bias appraisal. RESULTS: Our systematic review included 121 studies (15,832 patients). Two studies demonstrated that virtual cognitive assessments could diagnose dementia with good reliability compared with in-person cognitive assessments: weighted kappa 0.51 (95% confidence interval [CI] 0.41-0.62) and 0.63 (95% CI 0.4-0.9), respectively. Videoconference-based cognitive assessments were 100% sensitive and specific for diagnosing dementia compared with in-person cognitive assessments in a third study. No studies compared telephone with in-person cognitive assessment accuracy. The Telephone Interview for Cognitive Status (TICS; maximum score 41) and modified TICS (maximum score 50) were the only virtual cognitive tests compared with in-person cognitive assessments in >2 studies with extractable data for meta-analysis. The optimal TICS cutoff suggestive of dementia ranged from 22 to 33, but it was 28 or 30 when testing was conducted in English (10 studies; 1673 patients). Optimal modified TICS cutoffs suggestive of MCI ranged from 28 to 31 (3 studies; 525 patients). Sensory impairment was the most often voiced condition affecting assessment. CONCLUSION: Although there is substantial evidence supporting virtual cognitive assessment and testing, we identified critical gaps in diagnostic certainty.

COVID-19 related mental health issues: a narrative review of psychometric properties of scales and methodological concerns in scale development.

OBJECTIVES: The global crisis of COVID-19 and its consequential strict public health measures placed around the world have impacted mental health. New scales and tools have been developed to measure these mental health effects. This narrative review assesses the psychometric properties of these scales and tools and methodological aspects of their development. METHODS: PubMed, PubMed Central, and Google Scholar were searched for articles published from 15 May 2020 to 15 August 2020. This search used three groups of terms ("tool" OR "scale" AND "mental" OR "psychological"; AND "COVID-19" OR "coronavirus"). The identified scales were further evaluated for their psychometric properties and methodological aspects of their development. RESULTS: Though the studies developing these scales (n = 12) have demonstrated their robust psychometric properties, some methodological concerns are noteworthy. Most of the scales were validated using internet-based surveys, and detailed descriptions of the mode of administration, sampling process, response rates, and augmentation strategies were missing. CONCLUSIONS: The heterogeneous and inadequate reporting of methods adopted to evaluate the psychometric properties of the identified scales can limit their utility in clinical and research settings. We suggest developing guidelines and checklists to improve the design and testing, and result in reporting of online-administered scales to assess the mental health effects of the COVID-19 pandemic.

Assessment of Prorated Scoring of an Abbreviated Protocol for the National Institutes of Health Toolbox Cognition Battery.

OBJECTIVE: To evaluate an abbreviated NIH Toolbox Cognition Battery (NIHTB-CB) protocol that can be administered remotely without any in-person assessments, and explore the agreement between prorated scores from the abbreviated protocol and standard scores from the full protocol. METHODS: Participant-level age-corrected NIHTB-CB data were extracted from six studies in individuals with a history of stroke, mild traumatic brain injury (mTBI), treatment-resistant psychosis, and healthy controls, with testing administered under standard conditions. Prorated fluid and total cognition scores were estimated using regression equations that excluded the three fluid cognition NIHTB-CB instruments which cannot be administered remotely. Paired t tests and intraclass correlations (ICCs) were used to compare the standard and prorated scores. RESULTS: Data were available for 245 participants. For fluid cognition, overall prorated scores were higher than standard scores (mean difference = +4.5, SD = 14.3; p < 0.001; ICC = 0.86). For total cognition, overall prorated scores were higher than standard scores (mean difference = +2.7, SD = 8.3; p < 0.001; ICC = 0.88). These differences were significant in the stroke and mTBI groups, but not in the healthy control or psychosis groups. CONCLUSIONS: Prorated scores from an abbreviated NIHTB-CB protocol are not a valid replacement for the scores from the standard protocol. Alternative approaches to administering the full protocol, or corrections to scoring of the abbreviated protocol, require further study and validation.

Cognitive testing in the COVID-19 era: can existing screeners be adapted for telephone use?

Aim: To examine whether two existing cognitive screeners might be adapted for telephone administration by omission of item content requiring visual cues or assessment. Materials & methods: Data from a test accuracy study of Mini-Addenbrooke's Cognitive Examination (MACE) and Free-Cog were used to derive scores for 'Tele-MACE' and 'Tele-Free-Cog'. Results: As in the index study, both Tele-MACE and Tele-Free-Cog proved sensitive for dementia diagnosis. Tele-MACE had a better balance of sensitivity and specificity than observed with MACE. Tele-MACE was sensitive for mild cognitive impairment diagnosis, whereas Tele-Free-Cog was more specific for mild cognitive impairment. Conclusion: Existing cognitive screeners may be adapted for telephone administration. Such developments may prove necessary in the COVID-19 era as remote rather than face-to-face cognitive assessment increasingly becomes the established norm.

Pre-appointment online assessment of patient complexity: Towards a personalized model of neuropsychological assessment.

Recent events such as the global pandemic of COVID-19 have challenged neuropsychologists to scale up their capacity to conduct portions of their assessment remotely. While more complex patients will likely continue to require on-site, office-based interaction and assessment, the current emergency-based expansion of online and telehealth evaluation practices may ultimately lay the groundwork for more routine, online assessment of patients with less complex presentations in the future. To this end, the current study evaluated a pre-appointment, online methodology for differentiating referred pediatric patients based upon the scope and severity of their caregiver-reported adaptive, academic, attentional, behavioral, and emotional impairment. Prior to on-site assessment, parents/caregivers of 2197 children (Mean age = 10.0y, range = 4-19y, 62% male) completed an online developmental history form screening for symptoms of adaptive, attentional, learning, affective, and behavioral impairment; 71% of those children eventually underwent assessment. Using latent class analysis, the data supported a reproducible 4-class model consisting of groups of children at increased risk for: 1) severe multi-domain dysfunction; the "High Complexity" group, 30%, 2) behavioral-affective (but not academic) dysregulation; the "Behavioral Focus" group, 13%, 3) academic (but not behavioral-affective) problems; the "Academic and Inattention" group, 37%, and 4) patients with minimal clinical complexity; the "Low Complexity" group, 20%. Comparison of pre-visit classification with day-of-assessment standardized test scores supported the validity of patient subtypes. Moving forward, pre-appointment clarification of patient complexity may support efficient patient triage with regard to assessment modality (e.g., on-site or online) and length of appointment (e.g., comprehensive or targeted).

Validity of teleneuropsychology for older adults in response to COVID-19: A systematic and critical review.

Objective: Due to the recent COVID-19 pandemic, the field of neuropsychology must rapidly evolve to incorporate assessments delivered via telehealth, or teleneuropsychology (TNP). Given the increasing demand to deliver services electronically due to public health concerns, it is important to review available TNP validity studies. This systematic review builds upon the work of Brearly and colleagues' (2017) meta-analysis and provides an updated review of the literature, with special emphasis on test-level validity data.Method: Using similar methodology as Brearly and colleagues (2017) three internet databases (PubMed, EBSCOhost, PsycINFO) were searched for relevant articles published since 2016. Studies with older adults (aged 65+) who underwent face-to-face and TNP assessments in a counterbalanced cross-over design were included. After review, 10 articles were retained. Combined with nine articles from Brearly's analysis, a total of 19 studies were included in the systematic review.Results: Retained studies included samples from 5 different countries, various ethnic/cultural backgrounds, and diverse diagnostic populations. Test-level analysis suggests there are cognitive screeners (MMSE, MoCA), language tests (BNT, Letter Fluency), attention/working memory tasks (Digit Span Total), and memory tests (HVLT-R) with strong support for TNP validity. Other measures are promising but lack sufficient support at this time. Few TNP studies have done in-home assessments and most studies rely on a PC or laptop.Conclusions: Overall, there appears to be good support for TNP assessments in older adults. Challenges to TNP in the current climate are discussed. Finally, a provisional outline of viable TNP procedures used in our clinic is provided.